1. Field of the Invention
This invention relates to a picture signal encoding method or apparatus used in a system for encoding picture signals for moving pictures for storage on a recording medium, such as an optical disc, a magnetic disc or a magnetic tape or in a system for transmitting picture signals of moving pictures over a transmission route. This invention also relates to a signal recording medium used in these systems.
2. Description of Related Art
In general, digital video signals are extremely large in information volume. Therefore, if it is attempted to record the digital video signals for prolonged time on a small-sized recording medium with a small storage information volume (storage capacity), or to transmit the digital video signals over a transmission channel of limited communication capacity, it is indispensable to encode the digital video signals by high efficiency encoding in order to reduce the information volume. For meeting this demand, there are proposed high efficiency encoding systems exploiting the correlation of video signals. One of these systems is the Moving Picture Exerts Group (MPEG) system, in which the inter-frame difference is taken of video signals to lower redundancy along time axis and subsequently the orthogonal transform technique such as discrete cosine transform (DCT) is used to lower redundancy along the spatial axis for efficiently encoding the video signals.
With the MPEG system, the pictures making up a moving picture are classified into three sorts of pictures, namely I-pictures, P-picture or B-pictures, for compression coding of picture signals, as shown in FIG. 1. The I-picture is an intra-coded picture and is decoded independently of other pictures. The P-picture is a forward predictive coded picture obtained on prediction coding from the temporally backward or past I- or P-pictures. The B-picture is a bi-directional predictive coded picture obtained on predictive coding from forward, backward or bi-directional pictures using a temporally forward or backward I- or P-picture. A set of these pictures constitutes so-called group-of-pictures (GOP).
Each picture is divided into smaller units termed macro-blocks. In the case of, for example, a 4:2:0 format digital video, each macro-block MB is made up of six blocks, namely four luminance blocks (Y) and two spatially associated chrominance blocks Cb and Cr. Each of these blocks is made up of 8 by 8 pixels, totaling at 64 pixels.
If the number of pixels in the horizontal direction of the luminance signals is PH, the number MBH of the macro-blocks of a picture in the horizontal direction is given by the following equation (1): EQU MBH=(PH+15)/16 (1)
On the other hand, if the number of pixels in the vertical direction of the luminance signals is PV, the number MBV in the vertical direction of a picture of an interlaced frame (with a flag defined by MPEG2 progressive.sub.-- sequence=0) is given by the following equation (2): EQU MBV=2*((PV+31)/32) (2)
The number NMB in a picture is found from the following equation (3): EQU NMB=MBH*MBV (3)
FIG. 2 shows the coding sequence of each macro-block MB in an I-picture. That is, the macro-blocks MB in the I-picture are encoded sequentially (in the order specified by numerical figures entered in FIG. 2) beginning from the first or upper left macro-block up to the NMBth macro-block MB, as shown in FIG. 2.
In the MPEG coding system, the numbers of allocated bits are preferably varied adaptively from picture to picture for taking advantage of the performance of the three sorts of pictures (I- P- or B-pictures) of different properties. Also, in the MPEG coding system, the quantization scale for quantization of DCT coefficients of a macro-block MB can be varied from one macro-block MB to another for performing bit rate control so that the volume of generated bits of a coded picture will approach a designated amount of allocated bits.
Among widely known systems for controlling the quantization scale of a macro-block MB is a system disclosed in ISO/IEC JTC1/SC29/WG11/N0400 test Model 5, pp. 54 to 57 (April 1993). The corresponding technical information is afforded by, for example, U.S. Pat. No. 5,144,424 entitled APPARATUS FOR VIDEO DATA QUANTIZATION CONTROL. The control system for controlling the quantization scale of this macro-block MB is hereinafter termed TM5 rate control.
FIG. 3 shows a flowchart for this TM5 rate control.
In FIG. 3, assume that the number of pictures in a video sequence is NPICT and a variable specifying a picture for calculating the volume of allocated bits among the pictures of the sequence is pict.sub.-- i. That is, the flowchart of FIG. 3 shows the flow for calculation of the volume of allocated bits for the pict-ith one of the pictures in the sequence. The volume of allocated bits for the pict.sub.-- ith picture is BT. In the following explanation, each variable is described by the C++ language as one of programming languages.
First, at step ST51, the variable pict.sub.-- i is set to 1 as a picture for calculating the volume of allocated bits for specifying the first picture in the sequence.
After this step ST51, the above variable pict.sub.-- i is sequentially incremented by 1 in the processing from step ST52 to step ST63 for calculating the volume of allocated bits in each picture in the sequence. The processing from step ST51 up to step ST64 is repeated until the calculations of the volume of allocated bits come to a close for all pictures in the sequence.
For this processing from step ST52 up to step ST63, the volume of allocated bits BT for the pict.sub.-- ith picture is first calculated at step ST52. First, parameters specifying picture complexity degree are calculated from one picture type to another, and the volume of allocated bits of the GOP is allotted by proportionate distribution to each picture depending on the picture complexity degree.
At the steps ST53 and ST54 to ST60, the quantization scale of each macro-block in the pict.sub.-- ith picture is calculated. Here, the variable specifying the macro-block MB in which the quantization scale is calculated in the pict.sub.-- ith picture is set as mb.sub.-- i.
At step ST53, of the above steps ST53 up to ST60, the above variable mb.sub.-- i as the macro-block MB in the pict.sub.-- ith picture, in which to execute the calculations for the quantization scale, is set to 1 for designating the first macro-block in the picture.
After this step ST53, the above variable mb.sub.-- i is sequentially incremented by 1 in the processing from step ST54 up to step ST60, for calculating the quantization scale in each macro-block MB in the picture. The processing from ST54 to step ST60 is repeated until the calculations of the quantization scale for all macro-blocks MB in the picture come to a close.
As the processing from ST54 to step ST60, the total volume of the quantization scale in a picture under consideration is calculated, from the volume of bits occupying the virtual buffer, in accordance with the following equation (4): EQU GQ=d[mb.sub.-- i]*31/r (4)
where GQ is the global quantization scale.
In the above equation, d[mb.sub.-- i] is the volume of bits occupying the virtual buffer prior to coding of the mb.sub.-- ith macro-block MB, and r (termed a reaction parameter) is of a constant value pertinent to the target bit rate and the picture rate. The larger the volume of bits occupying the virtual buffer, the larger is the above global quantization scale GQ. Meanwhile, the calculations of the volume of bits occupying the virtual buffer will be explained subsequently in connection with the step ST58.
Next, at step ST55, a weighting coefficient W taking into account the visual characteristics of the mb.sub.-- ith macro-block MB is calculated from the following equation (5): EQU W=(act[mb.sub.-- i]+2*avg.sub.-- act)/(2*act[mb.sub.-- i]+avg.sub.-- act)(5)
In the equation (5), act[mb.sub.-- i is a minimum value of the variance of four luminance signal blocks of the macro-block MB. This value specifies picture pattern complexity degree of the luminance signals. The larger this value, the more complex is the picture pattern. In the above equation, avg.sub.-- act is a mean value of act[mb.sub.-- i]. In a flat portion where deterioration is perceptually acute, the weighting coefficient becomes larger. Conversely, in a picture portion with complex picture patterns, where deterioration is perceptually less acute, the weighting coefficient W becomes smaller. The weighting coefficient W is normalized so that its value ranges from 1/2 to 2 and an expected value of W in the I-picture will be equal to 1.
At the next step ST56, the quantization scale mQ of the mb.sub.-- ith macro-block MB is calculated in accordance with the following equation (6): EQU mQ=GQ/W (6)
At the next step ST57, the DCT coefficients of the mb.sub.-- ith macro-block MB are quantized and encoded for thereby calculating the volume of actually generated bits bit.sub.-- mb[mb.sub.-- i] of the mb.sub.-- ith macro-block MB.
At the next step ST58, the volume of bits occupying the virtual buffer is updated. dI, dP and dB are provided for the I-, P- and B-picture, respectively, and are updated in accordance with the following equations (7) to (9): EQU dI[mb.sub.-- i+1]=dI[1]+BG[mb.sub.-- i].multidot.BT*mb.sub.-- i/NMB(7) EQU dP[mb.sub.-- i+1]=dP[1]+BG[mb.sub.-- i].multidot.BT*mb.sub.-- i/NMB(8) EQU dB[mb.sub.-- i+1]=dB[1]+BG[mb.sub.-- i].multidot.BT*mb.sub.-- i/NMB(9)
In the above equations, dI[1], dP[1] and dB[1] denote the volume of bits initially occupying each virtual buffer and BG[mb.sub.-- i] denotes the sum of generated encoding bits bit.sub.-- mb from the first macro-block MB up to the mb.sub.-- ith macro-block MB of the picture, as shown in the following equation (10): EQU BG[mb.sub.-- i]=.SIGMA.bit.sub.-- mb[mb.sub.-- i] (i=1.about.mb.sub.-- i)(10)
In each of the above equations, (BT*mb.sub.-- i/NMB) denotes a value obtained on allocating the volume of bits BT allocated to a given picture to mb.sub.-- i macro-blocks MB among the total number of macro-blocks NMB of the picture.
At step ST59, the address of the macro-block MB is advanced by one. That is, the above variable mb.sub.-- i is incremented.
At the next step ST60, it is judged whether or not processing of the last macro-block of the pict.sub.-- ith picture has come to a close, that is whether or not the variable mb.sub.-- i is larger than the number of macro-blocks NMB in the picture. If the result is negative, that is if processing is not terminated, processing reverts to step ST54. If the result is affirmative, that is if processing has been terminated, processing transfers to the next step ST61.
At this step ST61, the variable pict.sub.-- i is incremented by 1 before proceeding to processing the next picture.
At step ST62, the volume of bits occupying the virtual buffers dI[NMB+1], dP[NMB+1] and dB[NMB+1] at the time of end of the above-described picture encoding are substituted for the volume of bits occupying the virtual buffers dI[1], dP[1] and dB[1]. That is, the initial volumes of the occupying bits of the virtual buffers are updated as shown by the equations (11) to (13): EQU dI[1]=dI[NMB+1] (11) EQU dP[1]=dP[NMB+1] (12) EQU dB[1]=dB[NMB+1] (13)
At the next step ST63, it is judged whether or not processing of the last picture of the sequence (with the number of pictures of NPICT) has come to a close. Specifically, it is judged whether or not the variable pict.sub.-- i is larger than the number of pictures NPICT of the sequence. If the result of judgment is negative, that is if processing has not come to a close, processing reverts to step ST52 and, if otherwise, that is if processing has come to a close, processing of TM5 rate control comes to a close.
Meanwhile, when calculating the quantization scale of the current macro-block MB in the above TM5 rate control, the quantization scale GQ is calculated from the current volume of bits occupying the virtual buffer as shown by the above equation (4). The global quantization scale GQ is calculated for stabilizing the volume of bits occupying the virtual buffer calculated by the above equations (7) to (9). That is, feedback is applied such that, if the volume of generated bits of the pictures up to now BG is smaller than the volume of bits allocated up to now (BT*mb.sub.-- i/NMB) such that the volume of bits occupying the virtual buffer is decreased (that is decreased in value), the global quantization scale GQ is diminished from the equation (4) whereby the volume of generated bits of the macro-block MB is increased to stabilize the volume of bits occupying the virtual buffers. If reverse is the case, that is if the volume of bits occupying the virtual buffer is increased in its value, there is applied such feedback that the global quantization scale GQ is increased to diminish the volume of generated bits of the macro-block MB to stabilize the volume of bits occupying the virtual buffers. This feedback control assures uniform volume of bits generated in the pictures.
However, this control raises a problem if the picture is not uniform in pattern, for example, if the picture contents differ in the upper and lower halves, such that the upper half is of a simple picture pattern whereas the lower half is complex in picture pattern. That is, since feedback is applied such that the volume of generated bits in the upper half will be equal to that in the lower half, the upper half picture is quantized finely, while the lower picture half is quantized roughly, with the result that the picture quality is not uniform. There is also raised a problem that when the macro-block MB transfers from the upper half picture to the lower half picture, the volume of bits occupying the virtual buffers tends to be non-stable.
Moreover, in the above TM5 rate control, the weighting coefficients taking into account the visual characteristics of the macro-blocks MB are calculated from the flatness of the luminance signals. This, however, is insufficient if desired to render deterioration in picture quality less acute.
In addition, in the TM5 rate control, stable rate control is not assured if the weighting coefficients W are not normalized so that the expected value in the I-picture of the weighting coefficients W taking into account the visual characteristics will be equal to 1. For example, assume that the quantization scale mQ indicated by numerical figures in FIG. 4 has been calculated in each macro-block MB, wherein smaller and larger values of the numerical figures stand for fine quantization and rough quantization, respectively. Also assume that, in this case, it is desired to reduce the quantization scale of the macro-blocks shown shaded in FIG. 4A, and the weighting coefficients only of the macro-blocks shown shaded in FIG. 4A are increased. Since the expected value of the weighting coefficients ceases to be equal to 1, only the quantization scale calculated for the macro-blocks following the macro-blocks shown shaded in FIG. 4B is increased.
In the TM5 rate control, since the weighting coefficients W for flatness of the macro-blocks MB are normalized relative to an average value of the flatness of the I-picture in calculations, the expected value of the weighting coefficients W is 1. However, if the weighting coefficients W are absolutely designated for the parameters taking into account visual characteristics of the macro-block MB, such that the expected value of the weighting coefficients W ceases to be 1, stable rate control is not assured.